In an augmented reality system, a user's view of the real world is enhanced with virtual computer-generated graphics. These graphics are spatially registered so that they appear aligned with the real world from the perspective of the viewing user. For example, the spatial registration can make a virtual character appear to be standing on a real table.
Augmented reality systems have previously been implemented using head-mounted displays that are worn by the users. However, these suffer from many ergonomic drawbacks, such as a narrow field-of-view, low resolution, ghosting, and issues with eye-accommodation as a result of wrong focal distances. Other augmented reality display techniques exploit large spatially aligned optical elements, such as transparent screens, holograms, or video-projectors to combine the virtual graphics with the real world. These allow for higher resolution and larger display area configurations, improved eye accommodation, and easier calibration. However, these display techniques require the user's head to be properly aligned with the display in order for the graphics to be correctly spatially registered.
For each of the above augmented reality display techniques, there is a problem of how the user interacts with the augmented reality environment that is displayed. Where interaction is enabled, it has previously been implemented using indirect interaction devices, such as a mouse or stylus that can monitor the hand movements of the user in six degrees of freedom to control an on-screen object. However, when using such interaction devices the user feels detached from the augmented reality environment, rather than feeling that they are part of (or within) the augmented reality environment.
The embodiments described below are not limited to implementations which solve any or all of the disadvantages of known augmented reality systems.